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Disclaimer 


Certain commercial entities, equipment, or materials 
may be identified in this presentation in order to 
describe an experimental procedure or concept 
adequately. Such identification is not intended to 
imply recommendation or endorsement by NIST, nor 
is it intended to imply that the entities, materials, or 
equipment are necessarily the best available for the 
purpose. 
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The Problem 

6n04%Ei'Hm3V is 23 taps 



Using Keyboard from Android Lollipop 
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Purpose 


■ Explore current state of usability and security 
metrics for passwords 

■ Assign strength metrics to passwords for which we 
already had usability metrics 

* How much entropy is lost as a result of 
permuting passwords to be easier to enter on 
mobile devices? 
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Usability Background 

Tap On, Tap Off 



Usability 


* Context of use 

■ Effectiveness 

■ Efficiency 

■ Satisfaction 



Usability: ISO 9241 


* “The extent to which a product can be used by 
specified users to achieve specified goals with 
effectiveness, efficiency and satisfaction in a 
specified context of use.” 



Usability: Context of Use 


“Users, tasks, equipment (hardware, software and 
materials), and the physical and social environments 
in which a product is used.” [ISO 9241] 


* Mobile vs. desktop context 





Usability: Effectiveness 


* “Accuracy and completeness with which users 
achieve specified goals.” [ISO 9241] 


■ Generally measured via error rates 
■ Password entry errors 
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Usability: Efficiency 


* “Resources expended in relation to the accuracy 
and completeness with which users achieve 
specified goals.” [ISO 9241] 

■ Generally measured via time on task 

* Password entry time 

* Number of keystrokes (taps) 
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Usability: Satisfaction 


■ “Freedom from discomfort, and positive attitudes 
towards the use of the product.” [ISO 9241] 

■ Generally measured via standardized or 
customized questionnaires 



Usability & Security Parallels 


■ Confidentiality 
* Integrity 
" Availability 


* Effectiveness 
» Efficiency 
■ Satisfaction 
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Password 

Security Background 

Tap On, Tap Off 



Attacks on Passwords 


■ Password guessing 


* Brute force 

* Intelligent guessing 


} We are only concerned 
with these classes of attacks 


Eavesdropping 
Social Engineering 
Physical attacks 
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Password Strength 


* Password strength is often expressed in terms of 
entropy 

* Note: Entropy is at most only loosely related to the 
use of the term in thermodynamics. 

» Entropy was originally defined by Claude Shannon 
in the 1 950s 
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Password Metric Groups 


■ Two password metric groups 

* Classified by how a password is created 

« user generated passwords 

■ system generated passwords 
(a.k.a. randomly generated) 


Password metrics measure only one of these 
groups 



Randomly Generated 
Password Metrics 


Shannon entropy formula: H = log 2 (B A L) 

* H = total entropy 

* B = number of characters to choose from 

* L = password length 

[Kuo, 2006] uses modified Shannon entropy 



Shannon Entropy Examples 


Password 

Entropy Estimate 

5c2'Qe 

39.33 

3.bH1o 

39.33 

a7t?C2# 

45.88 

m3)61fHw 

52.44 

p4d46*3TxY 

65.55 

q80<U/C2mv 

65.55 

d51)u4;X3wrf 

78.66 

6n04%Ei'Hm3V 

78.66 

m#o)fp A 2aRf207 

91.76 

4i_55fQ$2Mnh30 

91.76 
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User Generated Password 
Metrics 


“Guessing entropy” 

* Estimate of the average amount of work required 
to guess the password of a selected user 

* Uses Shannon entropy as a foundation 

« “Measures” password strength based on a 
ruleset 



User Generated Password 
Metrics 


* “Min-entropy” 

■ Difficulty of guessing the easiest single 
password to guess in the population 

* NIST specifies dictionary tests and password 
histories as heuristics to ensure at least 1 0 bits 
of entropy 



800-63 Entropy Heuristic 


■ From NIST SP 800-63-2: 

* 1 st character = 4 bits per character 

* 2 nd thru 8 th = 2 bits per character 

* 9 th thru 20 th = 1.5 bits per character 

* 21 + = 1 bit per character 

* Upper + lower + non-alphabetic = 6 bit bonus 

* Dictionary check = 6 bit bonus 



800-63 Min-Entropy Ruleset 


* Search a dictionary of at least 50,000 words for the 
password 

* If found, reject password 

■ Passwords that are detectable permutations of the 
username are not allowed 
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Our Research & Results 

Tap On, Tap Off 



Prior Work 

Tap On, Tap Off 
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Prior Work 


* Recent behavioral study on mobile password entry 

■ Participants had to learn, input, and recall 10 
random passwords 

■ Onscreen keyboard switching significantly 
increased input time and introduced errors 
[Greene, Gallagher, Stanton, & Lee, 201 4] 
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Measurement Granularity 


■ Password level 

* The entire password is either accepted or fails 

■ Character level 

* Multiple types of character errors 

(e.g., transposition, deletion, substitution) 

* Important to look at the nature and number of 
errors users make when inputting passwords 



Percent of Total Errors 


Tiny Keyboards = More Errors 
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Onscreen Keyboards 


QWERTYU I OP 
ASDFGHJKL 


Z X C V B N M 

123 ® Q space return 
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Screen Depth 1 


Q W E 


T Y U I OP 


A S 


F G H 


O 


X C V B N M <*) 


.7123 


space 


return 


Screen Depth 2 


1 234567890 


- / : ; ( ) $ & @ 



Screen Depth 3 






# % 


space 


Current Work 

Tap On, Tap Off 
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Methodology 


* Defined a password permutation 

* Divided characters in password into “classes” 

* Uppercase (U), lowercase (L), numbers (N), and 
symbols (S) 

* Group similar characters together 
■ Example: 


5c2'Qe is permuted to Qce52‘ 



Permutation and Tap Counts 


Original 

Password 

Permuted 

Password 

Length 

Taps: 

Original, 

Permuted 

Keyboard 

Changes: 

Original, 

Permuted 

Taps 

Saved 

5c2'Qe 

Qce52‘ 

6 

11,8 

4,1 

3 

m3)61fHw 

Hmfw361) 

8 

11, 10 

2,1 

1 

q80<U/C2mv 

UCqmv802</ 

10 

19, 15 

7,3 

4 

6n04%Ei'Hm3V 

EHVnim6043%' 

12 

24, 17 

9,2 

7 

m#o)fp A 2aRf207 

Rmofpaf2207#) A 

14 

24, 19 

10, 4 
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Password Collisions 


■ Multiple unique passwords can permute to the 
same password: 


p4d46*3TxY 
*Y6xpd344 
YpT d4x463* 
x46d4*Y3T p 




Our Results 

Tap On, Tap Off 
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Experiment 1 : Fan-Out 


How many passwords 
collide with the same user- 
friendly password? 



How Many Collisions? 


Length 

10 th Percentile 

90 th Percentile 

Average 

6 

120 

180 

159 

8 

840 

1680 

1329 

10 

5040 

25200 

12659 

12 

27720 

277200 

132492 

14 

360360 

3153150 

1438513 

16 

2402400 

40360320 

17187712 

18 

24504480 

514594080 

208414540 

20 

221707200 

6518191680 

2327087101 



Experiment 2: Entropy Loss 


How much entropy is lost by 
permuting passwords? 
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How Much Entropy Is Lost? 


Length 

10 th 

Percentile 

90 th 

Percentile 

Average 

Additional 

Letters 

6 

6.9 

7.5 

7.3 

2 

8 

9.7 

10.7 

10.4 

3 

10 

12.3 

14.6 

13.6 

3 

12 

14.8 

18.1 

17.0 

4 

14 

18.0 

21.6 

20.4 

5 

16 

21.5 

25.0 

24.0 

6 

18 

24.5 

28.9 

27.6 

6 

20 

27.9 

32.6 

31.2 

7 
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Experiment 3: All-Lowercase 


How much additional 
password length would we 
need to just change over to 
all lowercase letters? 
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What About All Lowercase? 


Complex 

Password 

All -Lowercase 

Extra Letters 

6 

9 

3 

8 

12 

4 

10 

14 

4 

12 

17 

5 

14 

20 

6 

16 

23 

7 

18 

25 

7 

20 

28 

8 
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q80<U/C2mv 

VS 

dmstpjnwqiwqok 



Unholster your 
phones and type this: 

m#o)fp A 2aRf207 



Now type this: 
Rmofpaf2207#) A 
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Recap 


■ Entering complex passwords on mobile devices is 
difficult 

■ Our password permutation makes it easier 

* We precisely measure the security loss 

* Fixed by adding a couple extra characters 
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Conclusions 


■ Device constraints matter 

* Old password policies play badly with new devices 

* Both usability and security must be considered 
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Code 


• https://github.com/usnistgov/PasswordMetrics 
■ https://github.com/usnistgov/DataVis 



Questions? 


* For additional research, visit NIST’s Information 

Technology Laboratory: 

* Kristen Greene 
Information Access Division 
nist.gov/itl/iad 

* John Kelsey 
Joshua Franklin 
Computer Security Division 
csrc.nist.gov 
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Extras 

Tap On, Tap Off 
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Data Viz Tool 

Tap On, Tap Off 
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Prior NIST Tool 


* Cathryn Ploehn’s SURF (Summer Undergraduate 
Research Fellowship) project 

* Shows usability and security metrics side-by-side 
for original and permuted passwords 

" Multiple levels of granularity 

* Filtering options 

■ https://github.com/usnistgov/DataVis 
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Password length: 6-14 



Show/Hide Columns 






Show Hide Columns 
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LPD 

per-rule and 

total scores keystrokes 
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Metrics for original 
password 


Metrics for permuted 
password 
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Original Password 

q856VW 

Symbol start: 0 

Number of Chunks: 


Number of Characters: 

2 

Unsentence-like 

capitalization: 

1 

Mixed Character String: 

1 

Pronounceable: 

0 

Total LP Difficulty: 

4 

# of Desktop 
keystrokes: 

11 

# of Android 
Keystrokes: 

11 

# of iPad Keystrokes: 

11 

entropy: 

39 


entropy 
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Permuted Password 

VWq856 

0 

Symbol start 

0 

Number of Chunks 

2 

Number of Characters 

1 

Unsentence-like 

capitalization 

0 

Mixed Character String 

0 

Pronounceable 

3 

Total LP Difficulty 

10 

# of Desktop keystrokes 

10 

# of Android Keystrokes 

10 

# of iPad Keystrokes 

33 

entropy 
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